Admission Control for Combined Guaranteed Performance and Best Eeort Communications Systems under Heavy Traac

Communications systems often have many types of users. Since they share the same resource, there is a con ict in their needs. This con ict leads to the imposition of controls on admission or elsewhere. In this paper, there are two types of customers, GP (Guaranteed Performance) and BE (Best E ort). We consider an admission control of GP customer which has two roles. First, to guarantee the performance of the existing GP customers, and second, to regulate the congestion for the BE users. The optimal control problem for the actual physical system is di cult. A heavy tra c approximation is used, with optimal or nearly optimal controls. It is shown that the optimal values for the physical system converge to that for the limit system and that good controls for the limit system are also good for the physical system. This is done for both the discounted and average cost per unit time cost criteria. Additionally, asymptotically, the pathwise average (not mean) costs for the physical system are nearly minimal when good nearly optimal controls for the limit system are used. Numerical data show that the heavy tra c optimal control approach can lead to substantial reductions in waiting time for BE with only quite moderate rejections of GP, under heavy tra c. It also shows that the controls are often linear in the state variables. The approach has many advantages. It is robust, simpli es the analysis (both analytical and numerical) and allows a more convenient study of the parametric dependencies. Even if optimal control is not wanted, the approach is very convenient for a systematic exploration of the possible tradeo s among the various cost components. This is done by numerically solving a series of problems with di erent weights on the costs. We can then get the best tradeo s, and the control policies which give them. Key words: admission control, control of communications systems, control of queueing networks, heavy tra c limits, ergodic control, singular control, weak convergence AMS numbers: 90B22, 60K25, 60K30, 60F17, 93E20, 93E25 2 1 Introduction Broadband-ISDN high speed networks allows the possibility of integrating different services in one single telecommunications network. In particular, they handle applications that require guaranteed QoS (quality of service), such as bounded delays bounded cell loss rates, and a guarantee on the throughput. Such a service is called Guaranteed Performance (GP). On the other hand, they support also more exible applications, such as data transfer, that are less sensitive to instantaneous variations in available bandwidth, in delays, jitter etc., and which do not require guarantees on throughputs or delays. We call the latter \Best E ort" (BE) tra c. In the context of ATM, this corresponds to the Available Bit Rate (ABR) service category [1], which can adapt to the bandwidth unused by the GP service classes. In context of the Internet, the BE tra c are the TCP/IP connections, which use a congestion avoidance mechanism [17] so as to adapt to the available bandwidth, in contrast with real-time applications that use UDP (UDP is a protocol which does not adapt its transmission rate to the current congestion state of the system) such as phone over Internet [5]. In particular, we suppose that the BE users share the remaining bandwidth, as for example, in the standard Internet. (Note that, unlike ATM, the Internet does not provide performance guarantees for real time-applications. However, in practice, the TCP/IP connections do adapt their transmission rate to the available bandwidth left over after the UDP sessions, see [3]). Since the GP and BE share the same resources of the network, there is a con ict in their needs, and the question of admission control arises. The purpose of this paper is to analyze the impact that Connection Admission Control (CAC) performed on GP tra c has on the BE tra c, in order to properly allocate resources in a dynamical manner as well as guarantee QoS. In ATM networks, a large part of the architecture has been standardized [1]. Implementation of the CAC is however not standardized and is left to the network manager. This has motivated much research and a large literature has emerged. Several approaches have been used in designing CACs. The rst approach is based on characterizing the input tra c of a source by some simple parameters, such as the e ective bandwidth; basically it estimates the bandwidth that should be given to a source to guarantee that the cell loss rate does not exceed (in some asymptotic sense) some small number. Examples of this approach are [10, 15, 13, 18, 19, 30, 31, 35]. A di usion based statistical CAC was proposed in [12]. A recent approach to admission control estimates (on-line) the consequence of possible admission, using real-time measures of the network activity [14, 33, 34]. An optimal-control approach for call admission, using a Markov Decision Processes) approach has been used in a number of papers [11, 16, 32]. All the cited papers focused on the impact of admission control on GP tra c. The CAC was not intended to be performed on BE tra c, since BE tra c is su ciently exible so as not to deteriorate QoS that are required by GP 3 connections. In particular, the ATM forum has decided [1] that ABR sessions will not be subject to CAC (unless they require from the network a guarantee on minimum cell rate). We shall therefore assume that BE sessions are always admitted into the network, unless some very large limit is reached. This limit might represent the number of sessions that can be handled simultaneously by the switches. This limit will not depend on the available bandwidth. Quite often, CACs are designed taking only into account the performance requirements of GP sessions. However, as already noted in [2, 4], under appropriate operating conditions in the network (in particular, the heavy-tra c conditions, which correspond to an e cient utilization of the resources), it is possible to improve substantially the performance (delays and throughputs) of the BE sessions by modifying slightly the CAC for the GP sessions. References [2, 4] focused on analyzing the performance of GP and BE sessions for given CACs (performed on GP sessions) that take into account also the performance of BE sessions. In this paper, we go one further step and provide a design based on an optimization approach. We consider the optimal admission control arising when the number of sessions and the available bandwidth are taken to be large, and scaled in an appropriate way so as to perform in the desirable heavy tra c region. The approach taken is that of optimal control in the heavy tra c regime. In this regime, the system has little spare bandwidth over what is needed to handle the \average load." It is shown that the physical system can be well approximated by a controlled (re ected) di usion process, and that good controls for this di usion process are good for the physical system under heavy tra c. The resulting controls are easily implementable (and are often simply linear in the state variables) and can give a signi cant improvement of the performance of the BE sessions by only a small rejection of GP sessions. The work is a continuation of the past work on the control of communications and queueing systems under heavy tra c conditions [28, 26, 25, 21, 29, 24, 27, 22]. The extensive numerical results in [24] illustrated the power of such an approach. It obtained excellent controls and system performance and obtained very useful information which would not otherwise be available. The paper [29], concerned with trunk line problems, also showed the power of the heavy tra c approach for modeling, simplifying and controlling very complex systems. The control problem for the actual physical model is quite hard, if not virtually impossible to solve. The system is not always Markovian. Even in the Markovian case, the number of states can be extremely large. Additionally, even for an uncontrolled Markovian system, the computation of the steady state bu er over ows and delays is di cult. The heavy tra c approach provides much analytical and computational simpli cation. The optimally controlled costs for the physical problem converge to the optimally controlled cost for the limit problem, and a nearly optimal control for the limit problem is also nearly optimal for the physical problem if the tra c is heavy. There are many other advantages to the combined heavy tra c and optimal 4 control approach. It yields the appropriate dimensioning of the system, and shows that good performance can be achieved with only modestly extra bandwidth (over average requirements). The \limit" variables can be interpreted as \aggregated" states. The analysis takes advantage of the laws of large numbers and central limit theorems that come into play as the size of the system grows. The \limit" or aggregate equations can be used to compute nearly optimal controls for the physical system, and to get good estimates of various measures of performance. The relative simplicity of the form of the heavy tra c limit process facilitates understanding the parametric dependencies. As in [24], we can obtain both qualitative and quantitative information which is often very hard to get otherwise . Otherwise one would need to study many particular cases of systems with di erent parameters and sizes whose relationships would remain obscure. The method allows convenient numerical approximations (see [23, 24]) whose complexity is much less than that of even the physical Markov systems. In fact, the basic Markov chain approximation method of [23] has been coded for problems of this type and is publicly available. 1 The format is robust: bursty data, priorities, time dependence, dependence of arrivals on the system state, and other useful extensions can be readily handled. An important question concerns the tradeo between gains in QoS for BE vs, losses for GP. The heavy tra c formulation allows a systematic exploration of this. One solves the optimization problem with various weights for the associated losses and costs, and computes the values of the individual components of the cost under the optimal controls. This yields the set of possible tradeo s, under good operating policies. One can then choose the control which get the best performance for one component, with a constraint on the others. Usually, optimization for its own sake is not the main interest. But the use of optimizing or optimal control methods for exploration of the possibilities is of greater interest and provides a powerful tool for design. The value of this approach was amply demonstrated in [24], and the large gains due to the use of feedback control were demonstrated. This is equally valid for the present problem. Indeed, numerical data shows that substantial gains (say, 30% or even much more) in BE delays can be obtained with only very modest levels of rejection of GP customers. The percent rejected depends on the system parameters and the arrival rates, and it goes to zero as the system grows, for any xed percentage gain in delay. The structure of the paper is as follows. The basic model is presented in Section 2, and the input-output equation is written in a way that is convenient for the heavy tra c analysis. In the basic model, the BE customers share the available bandwidth equally, whatever it is. The general methods used for the analysis of this case also apply with little modi cation to the subsequent cases. 1See the home page of the Lefschetz Center for Dynamical Systems, Brown University Dept. of Applied Mathematics home page http://www.dam.brown.edu/lcds.html. Select the software link to get the documentation and codes. 5 Section 3 deals with the weak convergence of the heavy tra c approximations and the discounted cost function as the tra c intensity approaches unity. Many of the ideas have been used in the references in various ways, despite the di erence in the problems considered. Because of this and to save space, we provide detailed outlines with references where possible. There is a new problem with \tightness" of the set of the (singular) controls, but we show that they can be approximated such that they are tight. Section 4 considers the case where there is an upper limit to the bandwidth that any one of the BE customers can use. This changes the dynamics and the scaling, but the previous analysis can be carried over. In order to illustrate the power and versatility of the approach, in Section 5 we consider extensions where there are several classes of BE customers, which might be allocated bandwidth in class dependent ways. There is a surprising degeneracy, which can be exploited to simplify the analysis and subsequent numerical work. Section 6 extends the basic model to a multicast case. Here there are two channels (any number could be used), each with its own class of GP customers, and own control. But the BE customers must be transmitted simultaneously on each of the channels. Many variations of the format are possible. The ergodic cost problem is dealt with in Section 7. In Section 7 we impose a constraint on the rate at which GP arrivals can be rejected, so that previous results on the ergodic cost problem can be exploited. The form of the heavy tra c limit equations implies that this constraint is not very restrictive and this is borne out by numerical data. But, in Section 8 we show that it is not a restriction. The optimalmean cost per unit time for the physical system converges to the optimal \ergodic" value for the limit system as the size of the system and time go to in nity in any way at all. Furthermore, the limit of the pathwise (not mean) average costs per unit time cannot be better than the optimal ergodic value for the limit system. The pathwise average costs for the physical system can be made to be arbitrarily close to this ideal limit by using a nice nearly optimal (for the limit system) control on the physical system, under heavy tra c. The pathwise results are important, since in any single application, we have just one sample path, and the mean values are less important than the pathwise values. Furthermore, a \nice" nearly optimal control for the limit system provides nearly optimal values for the physical system, under heavy tra c, in both a mean and pathwise sense. 2 Problem 1: The Basic Setup In this section, we set up the notation and evolution equations for the basic problem. The development for the more complex problem classes is similar. There are two classes of users, which we refer to as class 0 and class 1. Class 1 being the GP tra c and class 0 the BE tra c. As usual in the analysis of systems under conditions of heavy tra c, the mean service capacity is slightly 6 greater than the mean demand. The system is parameterized by a parameter N . Both the system capacity, excess capacity (over what is required for the average demands) and demand grow as N ! 1; with the relative excess capacity) going to zero. This will be formalized below. A consequence of the heavy tra c analysis is that these relationships represent good design in that very good performance can be obtained. A well designed heavy tra c operating regime implies that the system is not overbuilt and can well handle the demands placed on it. In this and in the next section, the bandwidth is normalized so that each member of class 1 requires one unit of bandwidth. The members of class 0 share whatever bandwidth is left. An applications paradigm is that each arrival is the work for a \session," whose work arrives essentially at once, and is bu ered on arrival. This models well the more general situation in which the input rate of packets within a session is not less than the transmission rate, so that when we allocate a given bandwidth to a session then this bandwidth is indeed used by it. In Section 4 we shall relax this assumption. The channel is time shared, with a guaranteed time going to the class 1 customers, and the remaining to the class 0 customers. Thus, there is no limitation on the rate at which work can be done on the set of class 0 customers except for the the available capacity. Let i;N l ; l = 1; : : : ; denote the interarrival times for the members of class i, and set E i;N l = i;N To conform with standard usage, we also de ne the normalized mean \rates" i;N = 1=N i;N : The service times for the members of class 1 are exponentially distributed with constant rate 1;N . Class 1 can be controlled in that any requested admission can be denied by the controller. The system is a \loss" system in that any customer denied admission disappears from the system. For analytic convenience, we suppose further that there is a constant B0 + > 0 such that no class 0 customers are admitted to the system if the current number of class 0 customers in the system is greater than pNB0 +: It follows from the heavy tra c limit theorems (and from the numerical data) that this is inconsequential for large enough B0 +: In fact, the condition is useful only to simplify some of the analytic details for the ergodic cost criterion, and it can be set equal to in nity for the discounted cost function. But we include the nite upper bound here to unify the development. Let F 0;N(t) denote 1=pN times the number of class 0 customers not admitted by time t. The time requirements for the members of class 0 are also exponentially distributed. But the rate at which a customer of class 0 departs from the system depends on the (random) resources available to it while it is in the system. The \conditional mean instantaneous departure rate" for a class 0 customer at time t is de ned to be 0;N times the bandwidth available to that customer at that time. I.e., if at time t, B(t) is the total bandwidth unused by members of class 1, and there are A(t) members of class 0 in the system, then the probability (conditioned on the data up to that time) of a single departure of a member of class 0 in the time interval [t; t+ ) is 0;NB(t) =A(t)+o( ); and the probability 7 of more than one departure in that interval is o( ). The set of interarrival times, and the service times for class 1 are assumed to be mutually independent. We assume that there is b > 0 (parameterizing the \excess capacity") such that the channel capacity is CN = N 0;N 0;N + 1;N 1;N + bpN: (2:1) Suppose that i;N ! i and i;N ! i, all positive. Thus, the mean arrival rates and the channel capacity are allO(N ), and the channel capacity is O(pN ) greater than the mean requirements. For appropriate b, this will be seen to be su cient for good behavior. We also make the innocuous assumptions that i;N l = i;N 2 ; l; i; N is uniformly integrable, (2:2) and that there are 2 i such that E "1 i;N l i;N #2 ! 2 i : (2:3) If the interarrival intervals for class i are constant, then 2 i = 0: If the arrival stream for class i is Poisson, then 2 i = 1: De ne the scaled mean arrival process Si;N (t) = 1 N [# of class i arrivals by time t] and the normalized and centered sum W i;N (t) = 1 pN [Nt] Xl=1 "1 i;N l i;N # ; where [Nt] denotes the integer part. As usual in heavy tra c scaling, all of the basic system variables are scaled by 1=pN . De ne X0;N (t) to be 1=pN times the number of members of class 0 in the system at time t, and set X1;N (t) = 1 pN # of class 1 in system at time t N 1;N 1;N : ThusX1;N (t) is the scaled number of class 1 customers centered about the mean, if there were no rejections, and an in nite channel capacity. De ne Ai;N (t) (resp., Di;N (t)) to be 1=pN times the number of arrivals (resp., departures) of class i by time t, and F 1;N(t) denotes 1=pN times the number of arrivals of class 1 up to time t which the controller did not admit. Thus, Ai;N (t) = pNSi;N (t): 8 The (non decreasing) control process F 1;N ( ) is assumed to be admissible in that it is (!; t) measurable and its value at time t depends only on the data which is available up to time t. The system balance equations are X0;N (t) = X0;N (0) + A0;N (t) D0;N (t) F 0;N(t); (2:4a) X1;N (t) = X1;N (0) + A1;N (t) D1;N (t) F 1;N (t) U1;N (t); (2:4b) where U1;N (t) is 1=pN times the number of class 1 customers that could not be admitted due the entire channel being occupied by class 1 customers. This last term will disappear in the limit. We will suppose that the set X1;N (0); N is tight (bounded in probability): (2:5) If this set is not tight, then there will be a long delay before the heavy tra c regime is entered. The set fX0;N (0)g is always tight, since B0 + < 1. It is also supposed that the initial condition is independent of the subsequent arrival times, and service times for class 1. In order to simplify the convergence proofs, one needs to put the input and output processes into a more convenient form. Representation of the input processes. Following a common practice in weak convergence analysis, we decompose the arrival process as Ai;N (t) = 1 pN NSi;N (t) Xl=1 1 = 1 pN NSi;N (t) Xl=1 "1 i;N l i;N #+ 1 pN NSi;N (t) Xl=1 i;N l i;N : (2:6a) Note that NSi;N (t) Xl=1 i;N l equals t minus the time since the last arrival before or at t. Thus, by (2.6a), Ai;N (t) = W i;N (Si;N (t)) + t i;NpN i;N (t) pN ; (2:6b) where i;N (t) is the time since the last arrival before or at t, divided by the mean interarrival interval. The sequence of processes i;N ( )=pN converges weakly to the \zero" process. It does not a ect any of the subsequent calculations and for the sake of notational simplicity, it will be omitted in all of the subsequent system equations after (2.11). 9 Representation of the output processes. Owing to the exponential distribution of the service time for the class 1 customers, D1;N ( ) can be decomposed into the sum of the integral of the instantaneous conditional mean rate at which D1;N ( ) increases, and a martingale process ~ Di;N ( ). To do this, rst note that the conditional mean instantaneous rate of increase of D1;N ( ) at t is 1;N=pN times the number of class 1 customers in the system at t. We have the decomposition D1;N (t) = 1;N Z t 0 X1;N (s) +pN 1;N 1;N ds+ ~ D1;N (t): (2:7) The quadratic variation process associated with the martingale is just 1=pN times the integral in (2.7); namely, < ~ D1;N > (t) = 1;N pN Z t 0 X1;N (s) +pN 1;N 1;N ds: (2:8) The factor 1=pN appears due to the de nition of Di;N (t) as 1=pN times the number of departures by time t. Similar decompositions were used in [21, 26]. The departure process for class 0 is similarly decomposed into the integral of the conditional mean instantaneous rate at which D0;N ( ) increases, and a martingale ~ D0;N ( ). Let I0;N (t) denote the indicator function of the event that there are class i customers in the system at t. In preparation for this, rst note that the available bandwidth per class 0 customer at time t is CN hpNX1;N (t) + N 1;N= 1;Ni pNX0;N (t) I0;N (t); which equals N 0;N= 0;N + bpN pNX1;N (t) pNX0;N (t) I0;N (t): Thus the conditional mean instantaneous rate at which D0;N ( ) increases at t is pN 0;N 0;N + b X1;N (t) I0;N (t): Hence, D0;N (t) = 0;N Z t 0 pN 0;N 0;N + b X1;N (s) I0;N (t)ds + ~ D0;N (t); (2:9a) which we rewrite as D0;N (t) = 0;N Z t 0 pN 0;N 0;N + b X1;N (s) ds+ ~ D0;N (t) Y 0;N (t): (2:9b) 10 The term Y 0;N ( ) is a re ection term; it corrects for the di erence in the integrals in (2.9a) and (2.9b), and it can increase only when X0;N (t) = 0. The quadratic variation process associated with ~ D0;N ( ) is < ~ D0;N > (t) = 0;N pN Z t 0 pN 0;N 0;N + b X1;N (s) I0;N (s)ds; which is written more conveniently as 0;N Z t 0 0;N 0;N + b pN X1;N (s) pN I0;N (s)ds: (2:10) Now, putting all of the above representations together and canceling the pN i;N terms yields the forms: X0;N (t) = X0;N (0) +W 0;N (S0;N (t)) 0;N Z t 0 g0(XN (s))ds ~ D0;N (t) F 0;N (t) + Y 0;N (t) 0;N (t) pN ; (2:11a) X1;N (t) = X1;N (0) +W 1;N (S1;N (t)) 1;N Z t 0 X1;N (s)ds ~ D1;N (t) F 1;N (t) U1;N (t) 1;N (t) pN ; (2:11b) where we de ne g0(x) = b x1; (2:12) and the Y 0;N (t) term compensates for the fact that there are no departures of class 0 customers at time t if X0;N (t) = 0: It assures that X0;N (t) will stay non negative. Comments on weak convergence. The path space for all of the random processes is Dk[0;1), the space of functions which are right continuous, have left hand limits and take values in Euclidean k space for appropriate integers k. The Skorohod topology is used [6, 9]. This is the most common and convenient choice in heavy tra c analysis. The following is a convenient criterion for tightness in this space. It will be used implicitly. Let fY n( )g be a sequence of processes with paths in D[0;1), with probability one. Let T n(t) denote the stopping times with respect to the ltration engendered by Y n( ) and which are no larger than t. If lim !0 sup n sup 2T n(t)E (1 ^ jY n( + ) Y n( )j) = 0; (2:14) for each t, and fY n(t) : n; tg is tight; (2:15) then fY n( )g is tight [9]. 11 3 Discounted Cost Function and Weak Convergence The convergence theorem implies that only a bandwidth excess of order O(pN ) is needed for good performance. Although the set of applications is new, the setup so far is similar to that of many other problems of control of queues under heavy tra c. See, for example, [28, 21, 24, 22]. The main new questions concern the control functions F 1;N( ). We will be concerned with two types of cost functions; the discounted and the average cost per unit time (to be called the ergodic cost function). Until Section 7, we concentrate on the discounted problem. Much of the analysis carries over to the ergodic case with little change. Also, analogous procedures are used for the convergence proofs for all of the problem formulations in the following sections. Let > 0; ci > 0; where can be as small as we wish, and let k( ) be a non negative continuous function with k(0) = 0: The discounted cost function is de ned by CN (x; F 1;N) = E Z 1 0 e sk(X0;N (s))ds +E Z 1 0 e sXi cidF i;N(s): (3:1) The second term penalizes the rejections. We do not penalize the loss U1;N( ), since it is zero in the limit as N !1 no matter what the controls are. The rst term can be quite general. If k( ) is linear, then it simply penalizes the waiting time for class 0 customers. More generally, it can be non linear. For example, it might be zero for small values of the argument (if the delays at small values are considered to be unimportant), or it might increase superlinearly to discourage long delays. The allowed generality of the cost function is one of the advantages of the approach. De ne V N (x) = infF1;N CN (x; F 1:N); where the inf is over the admissible controls. The basic structure of the convergence proofs is similar to those in the cited references to controlled queues in heavy tra c, except for the questions of tightness and approximation of the control functions. After stating the convergence theorems, the proofs will be outlined and references given for many of the details. De ne Bi;N ( ) = W i;N (Si;N ( )) ~ Di;N ( ): Theorem 3.1. Let > 0 be small but arbitrary, and let F 1;N( ) be optimal controls. Suppose that fF 1;N( ); Ng is tight2 . Then the set Xi;N ( ); Bi;N ( ); F i;N( ); i = 1; 2;Y 0;N ( ); N 2By Theorem 3.3, this tightness assumption entails no loss of generality. 12 is tight. The weak sense limit of any weakly convergent subsequence satis es X0(t) = X0(0) 0 Z t 0 g0(X(s))ds + B0(t) + Y 0(t) F 0(t); (3:2a) X1(t) = X1(0) 1 Z t 0 X1(s)ds + B1(t) F 1(t); (3:2b) where the Bi( ) are mutually independent Wiener processes with variance parameters i(1 + 2 i ): 0 X0(t) B0 +; and Y 0( ) is the re ection term at zero. F 0( ) is the re ection term at the upper bound. The other processes are non anticipative with respect to the Wiener processes. Comments on the proof. Details for similar results are in [28, 26, 21, 24, 22]. First, by the renewal theorem Si;N ( ) converges weakly to the deterministic (limit scaled mean arrival rate) process with values it. By Donsker's Theorem [6, 9], W i;N ( ) converge weakly to mutually independent Wiener processes with variance parameters 2 i . Hence the W i;N (Si;N ( )) converge to mutually independent Wiener processes ~ W i( ) with variance parameters i 2 i : The set of martingales f ~ Di;N ( ); i; Ng can be proved to be tight (their quadratic variation processes are bounded by a constant times t), and have continuous limits. With the above results available, the tightness of fX1;N ( ); U1;N( ); Ng can be readily proved. This tightness implies that the sequence U1;N ( ) has \zero" weak sense limits. The tightness of fX1;N ( )g also implies the tightness of fX0;N ( ); Y 0;N ( ); F 0;N( ); Ng: In fact, the tightness of the fF 0;N( )g can be shown even without tightness of the controls, since the controls only decrease X1;N ( ) (hence X0;N ( )) and F 0;N( ) is a re ection process at an upper boundary. Indeed, one can show that F 0;N ( ) is asymptotically continuous with probability one. If it were not asymptotically continuous then the asymptotic discontinuity and the properties of the other driving terms for X0;N ( ) would imply that asymptotically there is a jump to the interior of [0; B0 +] from the upper boundary, which is impossible, since the individual steps go to zero as N !1 and F 0;N( ) can increase only on the boundary. The tightness of fY 0;N ( ); Ng implies that I0;N ( ) can be non zero only on set whose Lebesgue measure goes to zero a N !1: Hence, I0;N ( ) has no asymptotic in uence on the quadratic variation of the martingale ~ D0;N ( ). Now, given the tightness, extract a weakly convergent subsequence, with the weak sense limits being denoted by Xi( ); etc. It can be shown that the weak sense limits ~ Di( ) are mutually independent Wiener processes which are independent of the ~ W i( ), and have the variance parameters i: The Wiener property of the limits of the ~ Di;N ( ) is proved as in [21, Theorem 3.1]. The mutual independence is a consequence of the independence conditions and the de nition of \conditional mean instantaneous rate" via a conditional expectation, and an analogous computation is in [21, Theorem 3.1]. The fact that (3.2) 13 holds follows from the weak convergence. The non anticipativenss properties are proved by standard \martingale" means as in [28, Theorem 5.1], [21, Theorem 3.1] or [22, Theorem 2.1] De nition. The discounted cost function for (3.2) is C (x; F 1) = E Z 1 0 e sk(X0(s))ds + E Z 1 0 e sXi cidF i(s): (3:3) De ne an admissible control F 1( ) for (3.2) to be a non decreasing process which is non anticipative with respect to the Wiener processes. If F 1( ) is absolutely continuous, with derivative u( ), then we say that u( ) is admissible if it is non negative, measurable and non anticipative. De ne V (x) = infF1 CN (x; F 1); where the inf is over the admissible controls. We say that F 1;N ( ) has a derivative which is bounded by R if for each t and s > 0, F 1;N (t + s) F 1;N (t) Rs + 1=pN . The 1=pN term is needed since F 1;N ( ) is piecewise constant with jumps 1=pN . Theorem 3.2. Let XN (0)) X(0) (weak convergence) Then V N (XN (0))! V (X(0)): (3:4) Proof. By Theorem 3.3, it can be assumed without loss of generality that for each xed > 0, there is some set of optimal controls for (2.11), (3.2), with uniformly bounded derivatives; hence the set is tight. Owing to the discounting, we can suppose that there is T <1 such that the F 1;N ( ) do not change after time T : It is su cient to work with this set below. Given > 0, let F 1;N ( ) be a sequence of optimal controls. Let XN (0)! X(0): Then use the weak convergence, Fatou's Lemma and the optimality of F 1;N( ) for each N to get that + lim inf N V N (XN (0)) lim inf N CN (XN (0); F 1;N) C (X(0); F 1) V (X(0)): (3:5) Since is arbitrary, (3.5) implies that lim inf N V N (XN (0)) V (X(0)): (3:6) Now we prove the reverse inequality to (3.5), namely, lim sup N V N (XN (0)) V (X(0)): (3:7) To do this we apply the following widely useful approach. Note that, for each initial condition, the distribution of the limit process X( ) depends on the pair 14 (F 1( ); B( )). For each xed > 0, we rst nd an optimal pair (F ;1( ); B ( )) for the limit process with the property that there is a sequence of admissible controls F ;1;N( ) for (2.11) such that (de ne B ;N ( ) as BN ( ) was de ned but under control F ;1;N( )) fXN (0); F ;1;N( ); B ;N ( ))g converges weakly to (X(0); F ;1( ); B( )) and also that CN (XN (0); F ;1;N)! C (X(0); F ;1) V (X(0)) + : (3:8) For any > 0, there is an F ;1( ) and a sequence of admissible controls fF ;1;N( )g satisfying the requirement (3.8). See, e.g., [27, Section 5]. The sequence of controls given by the reference is admissible and the F ;1;N(T ) are uniformly bounded. It does not necessarily have a bounded derivative, and might not even be tight in the Skorohod topology. A time transformation methods was used in the reference to circumvent the tightness problem. But the method used here is simpler in the current case, since Theorem 3.3 shows how to alter the control sequence (without loss of generality) so that we have bounded derivatives (hence tightness) with (3.8) holding (with perhaps replaced by 2 ): Now, (3.8) and the fact that (due to the non optimality of F ;1;N ( )) V N (XN (0)) CN (XN (0); F ;1;N) yields (3.4). Comment on the optimal controls. The comments on the form of the optimal control in Section 7 also hold for the discounted cost problem. In particular, numerical data show that there is a piecewise linear or nearly linear switching curve such that the optimal control is to reject above and accept below, with any decision allowed when on the curve. For large N , this control will be nearly optimal for the physical system. These comments on the shape of the switching curves are based on (very consistent) numerical computations, but not on proofs. Comment on tightness. An optimal sequence fF 1;N( ); Ng need not be tight in general. Let X1;N (t0) > 0. Consider the example of a control which rejects until x1;N(t) reaches the value zero, and then stops. Since the required time for X1;N ( ) to reach zero is of the order of X1;N (t0)=pN , the control clearly converges to a step function in an obvious way. But, owing to the fact that it increases in small steps (of size 1=pN ), the sequence is not tight in the Skorohod topology. There are many ways of dealing with this problem. We can adapt the time transformation method of [25, 23]. But, owing to the relative simplicity of the dynamics and cost function there is a simpler way, which avoids the extra notation and concepts. Theorem 3.3. It can be supposed that the controls in Theorems 3.1 and 3.2 are tight. In fact, it can be supposed that they have uniformly bounded derivatives for each > 0. 15 Proof. Fix > 0. Since B0 + < 1; fk(XN (t)); k(X(t));N; tg is uniformly bounded. 3 Thus, due to the discounting, there is T < 1 and a sequence of =2 optimal controls which do not change after time T : Note that the control which is identically zero has uniformly (in N ) bounded costs. We need only work with controls for which the sequence of costs is uniformly bounded. Hence, we can suppose that the =2 optimal sequence F i;N ( ) and the =2 optimal control F 1( ) do not change after time T : For large enough K < 1, the sequence de ned by F 1;N K ( ) = F 1;N( ) ^K will be 3 =4 optimal, and similarly for F 1;N K ( ) = F 1( ) ^K. To see this, note that whatever the controls in the previous paragraph are, the boundedness of the costs imply that sup N EF 1;N (T ) <1; EF 1(T ) <1: (3:9) Furthermore, lim K sup N P F 1;N(T ) F 1;N(T ) ^K 6= 0 = 0; limK P F 1(T ) F 1(T ) ^K 6= 0 = 0: (3:10) (3.9) and (3.10) and a straightforward analysis using Fatou's Lemma implies that the costs and systems associated with the use of F i;N( )^K (resp, F i( )^K for the limit system) are asymptotically (as K ! 1) no worse (uniformly in N ) than the costs for the original untruncated controls. Now that we know that there are 3 =4 optimal controls F 1;N( ); F 1( ) which do not increase after T and that are uniformly bounded, we can show that we can bound the derivative as well: More precisely, we can show that there are R <1 and optimal controls which satisfy: F 1 R(t + s) F 1(t) Rs; (3:11) F 1;N R (t + s) F 1;N R (t) Rs+ 1=pN; (3:12) for all t; s > 0. In particular, let F 1 R( ) denote the largest control which satis es (3.11), but is no greater than F 1( ), and let F 1;N R ( ) be the largest control which satis es (3.12) and which is no greater than F 1;N( ). SinceK <1 and R is as large as desired, any jump in F 1;N( ) can be reached by F 1;N R ( ) in an arbitrarily short (uniformly in N and in the realization) time afterwards Thus, excluding a set of measure which goes to zero (uniformly in N ) as R ! 1; F 1;N(t) F 1;N R (t) goes to zero (uniformly in N ) as R ! 1. This and the boundedness of the F functions imply that X1;N (t) X1;N R (t) is bounded and (excluding sets of arbitrarily small measure) converges to zero 3Even if B0 + = 1; by imposing a growth rate k(x0) = O(jx0j1+ ) for large x0, 0 < 1; and assuming that supN EjX0;N (0)j2 < 1, Ek(X0;N (t)) is at most O(t2) for large t, uniformly in the control (and analogously for the limit system. 16 uniformly in N . Similar remarks hold for F 1;N( ); F 1;N R ( ). These results imply that the costs converge as well. The 1=pN term appears in (3.12) since F 1;N( ) is piecewise constant with an increment of 1=pN at each rejection. 4 Upper Limit to the Bandwidth for the (BE) Sharing Customers In the model of the previous two sections, the class 0 customers shared the available bandwidth, whatever it was, and used all of it. In general, it might not be possible for all of the available bandwidth to be used. For example, there might be local restrictions on the rate at which data can enter the channel bu er (e.g., bounded modem speed, etc.). This possibility changes the problem a little, and we will indicate the few required adjustments. Such examples are further illustrations of the versatility of the approach. Suppose that the maximum bandwidth that any single class 0 customer can use is C0. The main di erence in the development concerns the departure process for class 0 customers and the structure of the appropriate cost function. We now rede ne X0;N (t) = # of class 0 customers at time t N 0;N=(C0 0;N ) pN : Note that now X0;N (t) is centered around a mean value, assuming that each class 0 customer uses exactly C0 units of bandwidth. In Sections 2 and 3, the number of class 0 customers in the system was O(pN ), and X0;N (t) measured that actual number, scaled by 1=pN . Now, the number in the system will be O(N ), and X0;N (t) measures the deviation from the mean number, scaled by 1=pN . We suppose that class 0 customers are rejected if X0;N (t) B0 +; where B0 + <1. Theoretical and numerical results show that this will have negligable e ect if B0 + is large. The martingale decomposition (2.9) remains valid, but the instantaneous conditional mean departure rate is di erent, being determined by whether or not the available capacity per class 0 customer is greater than C0: The conditional mean instantaneous rate at which D0;N ( ) increases at time t is 0;N pN [# of class 0 in system at t] min available BW at t # of class 0 in system at t ; C0 I0;N (t); 17 which equals I0;N (t) 0;N pN min[available BW at t; C0(# of class 0 in system at t)] = I0;N (t) 0;N pN min N 0;N 0;N + bpN pNX1;N (t); C0 0;NN C0 0;N +pNX0;N (t) = I0;N (t) 0;NpN + I0;N (t) 0;Ng1(XN (t); (4:1) where we de ne g1(x) = min b x1; C0x0 : (4:2) Now, analogously to what was done in Section 2, we can write the decomposition as D0;N (t) = 0;NpNt+ Z t 0 0;Ng1(XN (s))ds + ~ D0;N (t) Y 0;N (t); where the quadratic variation process associated with the martingale ~ D0;N ( ) is < ~ D0;N > (t) = Z t 0 0;N + 1 pN 0;Ng1(XN (s)) I0;N (s)ds: The dynamical equation is (2.11) with (2.11a) replaced by X0;N (t) = X0;N (0) +W 0;N (S0;N (t)) 0;N Z t 0 g0(XN (s))ds ~ D0;N (t) F 0;N (t) + Y 0;N (t); (4:3) and the limit equation is (3.2) with (3.2a) replaced by X0(t) = X0(0) 0 Z t 0 g1(X(s))ds + B0(t) F 0(t): (4:4) In the cost function (3.1), the function k( ) was assumed to be non negative. This made sense since X0;N (t) was non negative. Now, since X0;N (t) can take any sign, we suppose that k(x0) takes the sign of x0 and is zero if x0 = 0: Note that for large negative x0, the departure rate is essentially limited by the C0 limitation, and the control has little e ect. Also, suppose that jk(x0)j = O(jx0j); sup N E X0;N (0) 2 <1: (4:5) Under the given conditions, Theorems 3.1 to 3.3 hold. The savings in waiting time for the class 0 customers (for the controlled problem) are of the order of that of the model in Sections 2 and 3. But, since there are many more customers in the system at any time, the savings per 18 customer are less. We are essentially concerned with \marginal" savings, at a \marginal" cost. Comments on the proofs. The proofs outlined in Section 3 work here as well, with essentially the same details. The only di erences are due to the fact that in the present case the X0;N (t) are not bounded below. But in the proofs, the second moment bound sup N;t;F1;N E X0;N (t) 2 <1 (4:6) is used in lieu of the zero lower bound. (4.6) is proved by use of a dominating system. The second moments are bounded by a constant plus the second moments of the following system, which is de ned on the interval ( 1; 0], and the re ection term F 0;N ( ) now acts at the origin, and keeps the state non positive: X0;N (t) = X0;N (0)+W 0;N (S0;N (t)) 0;N Z t 0 X0;N (s)ds ~ D0;N (t) F 0;N (t): (4:7) The proof of (4.6) for the model (4.7) is done by a Liapunov function technique, and can be found in [21, page 771] 5 Extension. Several BE and GP Subclasses The developments in the previous sections can be extended to the case where there are multiple subclasses of any of the classes (and similarly for the models in the subsequent sections). We will illustrate only one of the many possibilities, working with the setup in Sections 2 and 3. Suppose that there are now two types of BE (class 0) customers, called class 01 and class 02, with parameters 0i;N ; 0i;N ; i = 1; 2: We suppose that the natural analogs of the conditions in Sections 2 and 3 hold. There is a surprisng and very useful degeneracy, which simpli es both the analytical problem and the numerical analysis. [Note to author: moved the paragraph that was here to the end of the section] Analogously to the formula (2.1), we let the channel capacity be CN = N "Xi 0i;N 0i;N + 1;N 1;N #+pNb: (5:1) The bandwidth available at time t for both subclasses 01 and 02 is Xi N 0i;N 0i;N pNX1;N (t) +pNb; b > 0: The processes are de ned analogously to what was done in Section 2. E.g., D0i;N ( ) is the number of departures of subclass 0i by time t, divided by pN . 19 Until otherwise noted, let us assume that the available bandwidth is shared equally among all class 0 customers, irrespective of the subclass. Then the total conditional mean instantaneous rate at whichD0i;N ( ) increases at t is (following the idea used in Section 2 and (assuming that there are class 0 customers in the system) 0i;N pN [# of subclass 0i in system at t] avail BW at t total # of class 0 in system at t ; which equals 0i;NX0i;N (t)Xj N 0j;N 0j;N pNX1;N (t) +pNb pNPjX0j;N (t) : (5:2) De ne aN = 01;N= 01;N Pj 0j;N= 0j;N ; a = lim N aN : (5:3) The system is degenerate in that if the costs are bounded in N , then the ratios X01;N (t)=(X01;N (t) +X02;N (t)) converge to a as N !1: Thus we need only analyze the system with class 1 and one of the subclasses 0i. Only an informal argument will be given. Let us examine the mean rates of increase of A0i;N( ) and D0i;N ( ). The \mean rate" at whichA0i;N ( ) increases ispN 0i;N :De ne BN =Pi[ 0i;N= 0i;N ]: Set aN (t) = X01;N (t) X01;N (t) +X02;N (t) : Using the fact that the available bandwidth is partitioned equally among all the class 0 customers, the conditional mean instantaneous rates at whichD0i;N ( ); i = 1; 2; resp., increase at t are, resp., hpNBN X1;N (t) + bi 01;NaN (t); hpNBN X1;N (t) + bi 02;N 1 aN (t) : The di erences of the dominant arrival and departure terms for the two subclasses are, resp., pN 01;N BN 01;NaN (t) ; (5:4a) pN 02;N BN 02;N 1 aN (t) : (5:4b) For the case of Section 2, the analogs of these terms have the value zero. Let > 0 be small. If, for large N , aN (t) 62 [ a ; a+ ], then (5.4a) implies that there is a large force (of the order of pN ) returning it to this interval. 20 Similarly, (5.4b) implies that (1 N (t)) must be very close to (1 a). The contribution of the non dominant terms is relatively small in comparison. The degeneracy situation is similar if there are more than 2 subclasses. Many interesting variations of the multiple subclass problem can be analyzed. For example, we might wish to alter the above formulation to allow each of the 0i subclasses a di erent fraction of the available bandwidth. More concretely, suppose that there are positive numbers ki such that for each unit of bandwidth allocated to a customer of subclass 01, we allocate k2=k1 units of bandwidth to each customer of subclass 02. Then the dominant term in the conditional mean instantaneous rate at which D0i;N ( ) increases at t is 0i;NkiX0i;N (t) BNN pN (k1X01;N (t) + k2X02;N (t)) : (5:5) Rede ne aN (t): aN (t) = k1X01;N (t) Pj kiX0i;N (t) : (5:6) Then, the di erence between the dominant input and output terms is (5.4), but with the new value of aN (t) used. Thus, we see that the new value of aN (t) is very close to a for large N . Note that the weighted fair-share for di erent types of BE customers in the above equations is the the one de ned by the ATM forum for sharing bandwidth amongABR users (see the Section I.3 in Appendix I \Implementation Examples on ABR Service" [1]). 6 Multicast: The Limit Dynamical Equations and The Discounted Cost Function Now, consider the case where there are two channels. There are three classes of customers. Class 0 is as in Section 2, but must be transmitted simultaneously and with the same instantaneous rate on both channels. Class i, i = 1; 2, is to be transmitted on channel i only. We make the natural analogs of the assumptions of Sections 2 and 3, de ning Ai;N ( ); Di;N ( ); Xi;N ( ); i = 0; 1; 2; etc., analogously to what was done there. Analogously to (2.1), the capacity of channel i is assumed to be Ci N = NXi i;N i;N + bipN; bi > 0: (6:1) Any number of channels and 0 subclasses could also be used, with an arbitrary assignment of the subclasses to the channels. 21 At time t, the bandwidth available for class 0 customers on channel i is N 0;N 0;N + bipN Xi;N (t)pN: Thus, the conditional mean instantaneous rate at which D0;N ( ) increases at t is determined by the channel with the largest available bandwidth and (analogously to what was done in Section 2) is 0;NpNX0;N (t) pN min2664N 0;N 0;N + b1pN X1;N (t)pN pNX0;N (t) ; N 0;N 0;N + b2pN X2;N (t)pN pNX0;N (t) 3775 I0;N (t): (6:2) This equals hpN 0;N + 0;Ng2(XN (t))i I0;N (t); (6:3) where we de ne g2(x) = min b1 X1;N (t); b2 X2;N (t) : (6:4) Thus analogously to what was done in Section 2, we can write D0;N (t) = 0;NpNt+ Z t 0 g2(XN (s))ds + ~ D0;N (t) Y 0;N (t); where the quadratic variation of the martingale is < ~ D0;N > (t) = Z t 0 0;N + 0;N pN g2(XN (s)) I0;N (s)ds: The analog of (2.11) is X0;N (t) = X0;N (0) +W 0;N (S0;N (t)) 0;N Z t 0 g2(XN (s))ds ~ D0;N (t) + Y 0;N (t) F 0;N(t); (6:6a) and for i = 1; 2; Xi;N (t) = Xi;N (0) +W i;N (Si;N (t)) i;N Z t 0 Xi;N (s)ds ~ Di;N (t) F i;N(t) U i;N (t); (6:6b) where the Y 0;N (t) term compensates for the fact that there are no departures of class 0 customers at time t if X0;N (t) = 0, and U i;N (t) compensates for the 22 class 1 arrivals lost due to a full system (when he entire channel is occupied by class 1 customers). The discounted cost function is still (3.1), but now the sum has three terms. The analysis given in Section 3 holds here in the same way and the limit equations are X0(t) = X0(0) 0 Z t 0 g2(X(s))ds + B0(t) + Y 0(t) F 0(t); (6:7a) X1(t) = X1(0) 1 Z t 0 X1(s)ds + B1(t) F 1(t); (6:7b) where the Bi( ) are mutually independent Wiener processes with variance parameters i( 2 i + 1): The associated cost function is (3.3), and we still have V N (XN (0))! V (X(0)); (6:8) if XN (0)) X(0): The model of Section 4. Now, suppose that each class 0 customer can use at most a bandwidth C0: Then de ne X0;N (t) as in Section 4, and let B0 + <1: The development is a combination of those of Sections 3 and 4. Now, the conditional mean instantaneous rate at which D0;N ( ) increases at t is obtained as the minimum of three terms, depending on whether the available capacities in channels 1, 2, or C0 is the limiting factor. It is 0;N pN [# of class 0 in system at t] min avail BW in ch 1 at t # of class 0 in system at t ; avail BW in ch 2 at t # of class 0 in system at t ; C0 I0;N (t): (6:9) De ne g3(x) = min b1 x1; b2 x2; C0x0 : (6:10) Then (6.9) can be written as hpN 0;N + 0;Ng3(XN (t))i I0;N (t): All of the previous results continue to hold with g3( ) replacing g2( ). 7 The Ergodic Cost Function: The Basic Model: Bounded Control Rate For concreteness, we work with the system model and assumptions of Sections 2 and 3, although all of the results hold for all of the other models. In this section, 23 we will suppose that the controls F 1;N( ) and F 1( ) (for the limit system) have bounded derivatives in the sense used in Theorem 3.3, as follows: There is a constant R, which can be as large as we wish, such that _ F 1(t) R for all t, and for all t; s > 0, F 1;N(t + s) F 1;N(t) Rs + 1=pN: Thus, the maximum \rate\ of refusing admission to class 1 customers is bounded by pNR. The reasonableness of the bounded derivative assumption is also seen from the form of the limit equation (3.2), which (informally) suggests that one loses very little by bounding the derivative of F 1( ). Furthermore, it is completely borne out by our numerical data. The next section shows that we can make this assumption in the proofs with no loss of generality. Although useful in applications, the mathematical reason for the assumption of bounded control \derivatives" concerns the mathematics of the ergodic cost problem. Little is known about the ergodic cost problem for the limit system when the control functions are arbitrary right continuous functions. But a great deal is known when they have uniformly bounded derivatives. In that case, for the current non degenerate model (3.2), there is an optimal feedback control which is time independent and the optimal value R(x) (de ned below) does not depend on x. More importantly, for our purposes, for any > 0; there is an optimal time independent feedback control F ;1( ) such that u ( ) = _ F ;1( ) is arbitrarily smooth, and under which there is a unique stationary measure. The F ;1( ) plays the role of the F 1; ( ) in Theorem 3.2. The basic convergence results are quite technical. They are in [20] for the unconstrained (no re ecting boundaries) problem, with extensions to the constrained problem being in [21, 22]. Indeed, under our basic setup, the needed convergence results can be obtained from [22] by appropriate identi cation of terms. De ne the cost functions CN (XN (0); T; F 1;N) = Z T 0 k(X0;N (s))ds +Xi ciF i;N(T ); N R (XN (0); T ) = inf F1;N EC(XN (0); T; F 1;N)=T; N R (XN (0)) = lim sup T N R (XN (0); T ): For the limit system, de ne the analogous quantities, with the N dropped. If there is no rate R restriction, we drop the subscript R. We also suppose that (with little loss of generality)sup N EjX1;N (0)j2 <1: (7:1) In Section 3, it was shown that, for the discounted cost problem and large enough R, we can get as close to optimality as we wish. The proof in Theorem 3.3 used the fact that the discounting implied that we need concern ourselves only with a nite time interval. The proof is more subtle for the ergodic cost problem, and is given in the next section. 24 The results in [22] will apply if we have tightness of the doubly indexed (both t and N are indices now) set of processes XN (t + ); BN (t + ) BN (t); F 1;N(t+ ) F 1;N(t);N; t : (7:2) The tightness holds for the set of F 1;N (t + ) F 1;N(t) processes by the assumption on boundedness of the derivative. The set fW i;N(Si;N (t+ ));N; tg is tight due to the independence properties of the interarrival intervals and (2.2). A Liapunov function argument and the \derivative" restrictions on the controls can be used to prove that sup t;N;F1;N E X1;N (t) 2 <1: (7:3) The tightness holds for the set of ~ Di;N (t + ) ~ Di;N (t) processes by a direct application of the criterion (2.14) and use of (7.3). The above facts, the boundedness of fX0;N (t);N; tg, and the criterion (2.14), (2.15) are then used to get the tightness for the set fXi;N (t+ );N; tg. Given the tightness of (7.2) and the non degeneracy of the limit system (3.2) (the set of driving Wiener processes is non degenerate, in fact the components are mutually independent with positive variances), the following results follow by a direct application of the results and ideas in [22]. N R (XN (0); T )! R; (7:4) as T !1 and N !1 in any way at all, where R is the in mum of the costs for the limit system over controls with derivatives bounded by R, and it does not depend on the initial condition. Furthermore, for any > 0; lim N;T P CN (XN (0); T; F 1;N) T R = 0; (7:5) where N; T can go to their limits in any way at all, and F 1;N ( ) is an arbitrary sequence of controls. There is a converse to (7.5), which says that a good control for the limit system is a good control for the physical system. Given > 0; let F ;1( ) be an optimal control with smooth derivative u ( ) and an adaptation F ;1;N( ) to the physical system such that lim N;T P CN (XN (0); T; F ;1;N) T R + 2 = 0; (7:6) The results of the next section imply that we can replace R in (7.5) and (7.6) by The control u ( ) can be adapted for use on the physical system in many ways, for large N . For example, by rejecting an arrival of class 1 at t with 25 probability (conditioned on the past system data) u (XN (t))=[ 1;NpN ]. Alternatively, we need not have the rejection choices being random, provided that pN times the number rejected when the state is \near" x converges to u (x) as N !1: Comments on the controls. Numerical data show that the derivative _ F (t) = u(t) of the optimal control takes either the value R or zero, with the regions separated by a piecewise linear or nearly linear switching curve. One applies this control to the physical system as in the last paragraph. [This procedure is asymptotically equivalent to rejecting all arrivals when the state is above the switching curve.] Equation (7.6) holds for such discontinuous controls as well. This is important in applications since such controls are easily implemented. Numerical data show that the switching curves converge nicely to piecewise smooth (or even linear) curves as R ! 1. (7.6) holds for this curve a well. Then we reject all arrivals of class 1 when the state is above the switching curve. Analogous remarks hold for the discounted cost problem. Comment. Note that both (7.5) and (7.6) deal with pathwise average costs, not with average costs. Since any application is a single realization, the convergence of pathwise average costs is more important than the convergence of expectations. The inequalities (7.5) and (7.6) say that for large N , the optimal controls for the physical problem are (asymptotically) only negligibly better than the use of a nice almost optimal control for the limit system. Finally, we simply note without further comment that the methods in [20, 21, 22] can be adapted to prove that limN; !0 V N (XN (0)) = : 8 The Ergodic Cost Problem: The Basic Model: Arbitrary Controls Now, return to the problem of bounded derivative controls. We show that we can approximate any optimal or nearly optimal control by a control with which has a \derivative" bounded by R, for large enough R. In particular: Theorem 8.1. lim R [ R] = 0: (8:1a) lim R lim sup N N (x) N R (x) = 0: (8:1b) Proof. A detailed outline of the steps will be given. Unlike as in Theorem 3.3, we can not restrict ourselves to a nite interval. We need to show that for any > 0, there is R < 1 such that there is a optimal for both the physical and the limit system with bounded rate R : 26 The development proceeds in several steps. The steps will be outlined (informally to save space) for the physical system. The details are a little simpler for the limit system. 1. Given > 0; show that there is a B <1 such that the optimal cost will change by no more than if we do not reject when X1;N (t) < B : 2. Let > 0: Allowing only controls which do not reject if X1;N (t) < B, for some given 0 < B < 1; show that there is K < 1 such that if we further restrict the controls such that the increments F 1;N (n+1) F 1;N(n) are at most K for all n;N , then the optimal cost will change by no more than : 3. Let > 0. Allowing only with controls satisfying the restrictions of the rst two steps for some nite B;K, show that there is R < 1 such that the optimal cost will change by at most if we further restrict the controls to have maximum \derivative" R : Step 1 is the least di cult to accept even without a proof, since it is quite reasonable that there is a B < 1 such that an optimal or nearly optimal control that would not reject if X1;N (t) B: The proof is a formalization of the following idea. Given a control F 1;N( ) and a B > 0, de ne another control F 1;N B (t) F 1;N (t) where F 1;N B (t) is as close as possible to F 1;N (t), but acts only when X1;N (t) B: For large B, the change in X0;N ( ) is slight. To save space, we concentrate on the outline for the other steps. Thus, we start by supposing that there is 0 < B <1 such that there are no rejections if X1;N (t) B: We will show that, given > 0; there is K < 1; such that we lose less than in the cost if we restrict the control to satisfy F 1;N(n+ 1) F 1;N (n) K for all N;n; !. Owing to the fact that we do not reject ifX1;N (t) B; a Liapunov function argument can be used to get that there is C <1 such that sup N;t;F1;N E X1;N (t) 2 C: (8:2) Also, the same B restriction and (8.2) can be used to show that sup n;N;F1;N E F 1;N(n+ 1) F 1;N (n) 2 <1: (8:3) The sup in (8.2) is over all controls satisfying our B restriction. The proof of (8.3) computes a worst case on each interval, which is a control taking X1;N (n) satisfying only (8.2) to B as quickly as possible, then keeping it there until the end of the interval, and repeating on the next interval, etc. The uniform mean square boundedness of the part due to keepingX1;N ( ) at B on [n; n+1] follows from the re ection mapping and the mean square bounds on the martingales driving (2.11b). For the re ection mapping and the Lipschitz continuity of the re ection term as a function of the driving processes, see [8],[7, Proposition 2.1]. Given any F 1;N( ) satisfying our restriction, we proceed to approximate it by bounding the increments by K. The approximation will be denoted by 27 F 1;N K ( ); and the associated processes denoted by XN K ( ): De ne F 1;N K ( ) such that it satis es the restriction F 1;N K (n+1) F 1;N K (n) K , it is no greater than F 1;N( ); and tries to keep X1;N K ( ) as close as possible to X1;N ( ): We always have X1;N K (t) X1;N (t), hence X0;N K (t) X0;N (t): It is not hard to see that, for large enough K, X1;N K ( ) will repeatedly catch up to and equal X1;N ( ). We can decompose time into successive intervals where X1;N (t) < X1;N K (t), and where X1;N (t) = X1;N K (t). The key to the proof of step 2 is the observation that, as K ! 1; a larger percentage of time will be taken up by the latter intervals. More precisely, for any T0 <1; it can be shown that lim K lim sup N sup t;F1;N P nX1;N (s) 6= X1;N K (s) for some s 2 [t; t+ T0]o = 0: (8:4) (8.4) follows from the observations made before it. Choose the control F 1;N K ( ) as described. Then, starting at time t k for large k, the probability that X1;N K ( ) catches up to X1;N ( ) and equals it on [t; t+ T0] goes to unity as K !1. Note that if X0;N K (t) = 0; then X0;N (t) = 0: We next bound the \return times\ to the boundary x0 = 0: Indeed, it can be shown that lim T!1 lim sup N sup t;F1;N K ;K sup ! P nX0;N K (t+ s) 6= 0; for some s T data to to = 0: (8:5) This can be shown by a weak convergence argument, using the fact that it holds for the limit process, as follows. The worst case for proving (8.5) is where there is no control since the control only decreases X0;N (t). Thus, suppose that there are > 0, tn and Nn !1; Tn !1 such that (no control) lim n sup ! P X0;Nn (tn + s) 6= 0; for some s Tn data to tn : (8:6a) We will show a contradiction to (8.6a). Actually, it is more direct to show that the assertion lim n sup ! P Y 0;Nn(tn + Tn) Y 0;Nn(tn) = 0 data to tn : (8:6b) is false. The falsity of (8.6b) implies the falsity of (8.6a). A Lipaunov function argument using (7.1) and the fact that there is no control can be used to prove that sup n EjX1;Nn (tn)j2 <1: (8:7) Now, extract a weakly convergent subsequence of XNn (tn + ), and note that its limit X( ) satis es (3.2). The distribution of X1(0) depends on the selected convergent subsequence. But, owing to (8.7), EjX1(0)j2 is bounded uniformly 28 in the selected convergent subsequence. Using this last fact, the properties of (3.2) and the weak convergence now imply that (8.6b) cannot hold unless = 0: Now note that (a key point), if X0;N K (t) = 0 and X1;N (t) = X1;N K (t), then the two processes start again at t with equal initial values. The above results imply the following. For any > 0; with a probability arbitrarily close to one, the fraction of time that jX0;N (t) X0;N K (t)j on any time interval goes to zero as K !1; uniformly in the time interval and in (large)N: This implies that the change in the k( ) part of the cost can be made as small as desired by makingK large enough. By construction, F 1;N(t) F 1;N K (t); hence the control cost is no greater for the approximating control. We omit the details of the fact that E[F 0;N K (T ) F 0;N(T )]=T can be made as small as desired by making K large. But it can be proved by a weak convergnce argument and the facts established above. Thus for large enough K, we lose as little as desired by restricting F 1;N ( ) such that F 1;N(n + 1) F 1;N(n) K for all N;n. This completes step 2. Now, we turn to step 3 and make a few comments concerning the k( ) component of the cost. Given a control F 1;N( ) satisfying the restrictions of steps 1 and 2 (with constants B and K, resp.), nd a suitable approximation with a bounded \derivative". Let R denote the derivative bound. De ne a control F 1;N R ( ) with derivative bounded by R, such that F 1;N R (t) F 1;N(t); but where the associated process X1;N R ( ) is allowed to catch up with X1;N ( ) when possible. It will catch up repeatedly, for large enough R. This is because the maximum number of rejects on any time interval of unit length is 2K=pN . Since K is bounded and R large, except for an arbitrarily small time subinterval the number of rejects on any time interval [n; n+1] can be made as close as desired to what is needed, uniformly in N;n. Additionally, X1;N (t) X1;N R (t) is uniformly (in N; t) bounded. Thus, the values of X1;N (t) and X1;N R (t) will be arbitrarily close when X0;N R (t) (hence, X0;N (t)) hits zero, or very shortly thereafter (at most a time K=R+ O(1=pN ) later). Note that he approximation problem is more subtle than in step 2, since we cannot guarantee that X1;N R ( ) will equalX1;N ( ) on longer and longer intervals. [E.g., if F 1;N( ) jumps periodically, or if the limit is singular with respect to Lebesgue measure.] The following properties can be proved. First, by a Liapunov function argument, it can be shown that lim sup N sup t;R;F1;N R E X1;N R (s) 2 <1: 29 Using this, it can be shown thatlim supNsupn;R;F1;NR E supn s n+1 X1;NR (s) 2 <1;(8:8)with a similar estimate holding for the X1;N ( ): The above comments imply thatlimR lim supN supE supt T Z +tX1;N (s) X1;NR (s) ds = 0;(8:9)for any T < 1, and where are stopping times. Now use an argument basedon recurrence to X0;N ( ) to zero analogously to what was done in in step 2 toget that the k( ) costs are close for large R. Obviously, the component of thecost due to F 1;NR ( ) is no greater than that due to F 1;N ( ). Again, by a weakconvergence argument, it can be shown the over ow costs also converge, andthe details are omitted.9 DataSome typical data is given in the tables below. The cost function isc0EX0(1)+EF 1(1)+5EF 0(1), all stationary values. B0+ = 6:4; and larger values made littledi erence. The individual components of the cost function are tabulated, and wewrite EF 0(1) = 0 if it is less than 10 4: The fraction of lost class 1 customers isEF 1(1)=[1pN ], so it depends on N . The tables indicate the potential tradeo sbetween the time gained for class 0 and the lost class 1 customers.Table A: 1 = :5; 0 = 1; 1 = 1; 0 = 1; 21 = 1; 20 = 1; b = 2:5EX0(1) EF 1(1) EF 0(1) %savings % Rejection of class 1N=100 N=103 N=104no cont. .5550.0032na000c0 = 5.267.489052%4.891.55.489c0 = 10 .1841.01067%1.01.319.101Table B: 1 = :25; 0 = :25; 1 = :5; 0 = 1; 21 = 1; 20 = 1; b = 2:5EX0(1) EF 1(1) EF 0(1) %savings % Rejection of class 1N=100 N=103 N=104no cont. 1.570.002na000c0 = 5.4241.463073%1.463 .386 .1463c0 = 10 .2992.34081%2.34.740.23430 Table C: 1 = 1; 0 = 1; 1 = 1; 0 = 1; 21 = 1; 20 = 3; b = 2:5EX0(1) EF 1(1) EF 0(1) %savings % Rejection of class 1N=100 N=103 N=104no cont. .8530.004na000c0 = 5.557.750035%.750.237 .0750c0 = 10 .4151.77051%1.77.316.177In the above examples and in all other cases that we tested numerically,a considerable saving in the global performance is obtained. The price payedfor this saving is the rejection of class 1 customers. However, the fraction ofrejected class 1 customers is acceptable for large N. As is seen in the tables, itis of the order of 1% for N = 1000, and less than 0.5% for N = 10000. We thusconclude that for large systems operating at a heavy tra c regime, we may gainconsiderably in overall performance of the system at a the cost of rejection avery small fraction of GP calls.References[1] The ATM Forum Technical Committee,Tra c Management Speci cation,Version 4.0, af-tm-0056, April 1996.[2] E. Altman, D. Artiges and K. Traore, \On the integration of Best-E ortand Guaranteed Performance services", INRIA Research report No. RR-3222 , 1997.[3] E. Altman, F. Boccara, J. Bolot, P. Nain, P. Brown, D. Collange andC. Fenzy, \Analysis of the TCP/IP Flow Control Mechanism in High-Speed Wide-Area Networks", IEEE Conference on Decision and Control,December 1995, New Orleans, Louisiana, (invited paper).[4] E. Altman, A. Orda and N. Shimkin, \Bandwidth allocation for Guaran-teed versus Best E ort service categories" IEEE Infocom, San-Fransisco,California, USA, March 29 April 2, 1998.[5] J-C. Bolot and A. Vega-Garcma, \Control mechanisms for packet audioin the Internet," Proceeding of the IEEE Infocom'96., San Francisco, CA,pp. 232-239, April 1996.[6] P. Billingsley. Convergence of Probability Measures. John Wiley, NewYork, 1968.[7] H. Chen and W. Whitt, \Di usion approximations for open queueingnetworks with service interruption", Queueing Systems, Theory and Appl.13, pp335{359, 1993.31 [8] P. Dupuis and H. Ishii, \On Lipschitz continuity of the solution mappingto the Skorokhod problem, with applications" Stochastics and StochasticsRep., 35, pp.31{62, 1991.[9] S.N. Ethier and T.G. Kurtz. Markov Processes: Characterization andConvergence. Wiley, New York, 1986.[10] A. I. Elwalid and D. Mitra, \E ective Bandwidth of general Markoviantra c sources and admission control of high speed networks", IEEE/ACMTrans. on Networking, 1, pp. 329-343, 1993.[11] E. A. Feinberg and M. I. Reiman, \Optimality of randomized trunk reser-vation", Probability in the Engineering and Informational Sciences, 8, pp.463-489, 1994.[12] E. Gelenber, X. Mang and R Onvural, \Di usion based statistical calladmission in ATM networks", Performance Evaliation, 27 & 28, pp.411-436, 1996.[13] R. J. Gibbens and P. J. Hunt, \E ective Bandwidth for the Multi-TypeUAS Channel", Queueing Systems, 9, pp. 17-28, 1991.[14] M. Grossglauser and D. Tse, \A Framework for Robust Measurement-Based Admission Control", Proc. ACM SIGCOMM 97, Cannes, France,September 1997.[15] R. Gu erin, H. Ahmadi and M. Naghshineh, \Equivalent Papacity and itsapplication to bandwidth allocation in High-speed networks",[16] A. Hordijk and F. Spieksma (1989), \Constrained admission control to aqueuing system", Advances of Applied Probability 21, pp. 409-431.[17] Van Jacobson. \Congestion avoidance and control". ACM SIGCOMM 88,pages 273-288, 1988.[18] F. P. Kelly, \E ective Bandwidth at multi-class queues", QueueingSytems, 9, pp. 5-16, 1991.[19] G. Kesidis, J. Walrand and C.-S. Chang, \E ective Bandwidth for multi-class Markov uids and other ATM sources", IEE/ACM Trans. on Net-working, 1, pp. 424-428, 1993.[20] H.J. Kushner. Optimality conditions for the average cost per unit timeproblem with a di usion model. SIAM J. Control Optim., 16:330{346,1978.[21] H.J. Kushner. Control of trunk line systems in heavy tra c. SIAM J.Control Optim., 33:765{803, 1995.32 [22] H.J. Kushner. Heavy tra c analysis of controlled multiplexing systems.Queueing Systems, Theory and Appl., 1997. to appear.[23] H.J. Kushner and P. Dupuis. Numerical Methods for Stochastic ControlProblems in Continuous Time. Springer-Verlag, Berlin and New York,1992.[24] H.J. Kushner, D. Jarvis, and J. Yang. Controlled and optimally controlledmultiplexing systems: A numerical exploration. Queueing Systems, The-ory and Appl., 20:255{291, 1995.[25] H.J. Kushner and L.F. Martins. Numerical methods for stochastic singularcontrol problems. SIAM J. Control Optim., 29:1443{1475, 1991.[26] H.J. Kushner and L.F. Martins. Heavy tra c analysis of a data trans-mission system with many independent sources. SIAM J. Appl. Math.,53:1095{1122, 1993.[27] H.J. Kushner and L.F. Martins. Heavy tra c analysis of a controlledmulti class queueing network via weak convergence theory. SIAM J. onContral and Optimization, 34:1781{1797, 1996.[28] H.J. Kushner and K.M. Ramachandran. Optimal and approximately op-timal control policies for queues in heavy tra c. SIAM J. Control Optim.,27:1293{1318, 1989.[29] H.J. Kushner and J. Yang. Numerical methods for controlled routing inlarge trunk line systems via stochastic control theory. ORSA J. Comput-ing, 6:300{316, 1994.[30] R.Mazumdar, \On call admission control", 2nd. IFIP Workshop on Tra cManagement and Synthesis of ATM Networks, Montreal, September 24-26, 1997.[31] Z. Liu, P. Nain and D. Towsley, \ Exponential bounds with applicationto call admission." Journal of the ACM, Vol. 44, No. 3, May 19.[32] K. W. Ross and D. Tsang, \Optimal Circuit Access Control Policies inan ISDN Environment: A Markov Decision Approach," IEEE Trans. onCommunications, Vol. 37, No. 9, pp. 934-939, 1989.[33] D. Tse and M. Grossglauser, \Measurement-Based Call Admission Con-trol: Analysis and Simulation", Proc. IEEE Infocom '97, Kobe, Japan,April 1997.[34] J. Walrand, \Measurement and control of ATM networks", 2nd. IFIPWorkshop on Tra c Management and Synthesis of ATM Networks, Mon-treal, September 24-26, 1997. 33 [35] W. Whitt, \Tail probabilities with statistical multiplexing and E ectiveBandwidth in multi class queues" Telecommunication Systems, 2, pp. 71-107, 1993.34